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Modeling Item Responses When Different Sub; ects 
Employ Different Solution Strategies 

Abstract 

A model is presented for item responses when different 
examinees employ different strategies to arrive at their answers, 
and when only those answers, not choice of strategy or subtask 
results, can be observed. Using substantive theory to 
differentiate the likelihoods of response vectors under a fixed 
set of solution strategies, we model responses in terms of item 
parameters associated with each strategy, proportions of the 
population employing each, and the distributions of examinee 
parameters within each. Posterior distributions can then be 
obtained for each examinee, giving the probabilities that they 
employed each of the strategies and their proficiency under each. 
The ideas are illustrated with a conceptual example about response 
strategies for spatial rotation items, and a numerical example 
resolving a population of examinees into subpopulations of valid 
responders and random guessers. 
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Introduction 



The standard models of item response theory (IRT) , such as 
the 2-, and 3-parameter normal and logistic models, 
characterize examinees in terms of their propensities to make 
correct responses. Consequently, examinee parameter estimates 
are strongly related to simple percent-correct scores (adjusted 
for the average item difficulties, if not all examinees have been 
presented the same items). Item parameters characterize the 
regression of a correct response on this overall propensity toward 
correctness . 

These models lend themselves well to tests in which all 
examinees employ the same strategy to solve the items. 
Comparisons among estimates of examinees' ability parameters are 
meaningful comparisons of their degrees of success in implementing 
the strategy. Item parameters reflect the number or complexity of 
the operations needed to solve a given item (Fischer, 1973). 

The same models can prove less satisfactory when different 
examinees employ different strategies. The validity of using 
scores that convey little more than percent-correct to compare 
examinees who have used different strategies must first be called 
into question. And item parameters keyed only to a generalized 
propensity toward correctness will not reveal how a particular 
kind of item might be easy for examinees who follow one line of 
attack, but difficult for those who follow another. 
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Extensions of IRT to multiple strategies have several 
potential uses. In psychology, such a model would provide a 
rigorous analytic framework for testing alternative theories 
about cognitive processing (e.g., Carter, Pazak, and Kail, 1983), 
In education, estimates of how students solve problems could be 
more valuable than how many they solve, for the purposes of 
diagnosis, remediation, and curriculum revision (Messick, 1984), 
And even when a standard IRT model would provide reasonable 
summaries and meaningful comparisons for most examinees, an 
extended model allowing for departures along predetermined lines 
(e.g., malingering) would reduce estimation biases for the 
parameters in the standard model. 

In contrast to standard IRT models, and, for that matter, to 
the "true score" models of classical test theory, a model that 
accommodates alternative strategies must begin with explicit 
statements about the processes by which examinees arrive at their 
answers. For example, items may be characterized in terms of the 
nature, number, and complexity of the operations required for 
their solution under each strategy that is posited. 

The recent psychometric literature contains a few 
implementations of these ideas. Tatsuoka (1983) has studied 
performance on mathematics items in terms of the application of 
correct and incorrect rules, locating response vectors in a two- 
dimensional space based on an ability parameter from a standard 
IRT model and an index of lack of fit from that model. Paulson 
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(1985), analyzing similar data but with fewer rules, uses latent 
class models to relate the probability of correct responses on an 
item to the features it exhibits and the rules that examinees 
might be following. Yamamo::o (1987) combines aspects of both of 
these models, positing subpopulations of IRT respondents and of 
non-scalable respondents associated with particular expected 
response patterns. Samejima's (1983) and Embretson's (1985) 
models for alternative strategies are expressed in terms of 
sub tasks whose results are observed, in addition to the overall 
correctness or incorrectness of the item. 

The present paper describes a family of multiple-strategy 
IRT models that apply when each examinee belongs to one of a 
number of exhaustive and mutually-exclusive classes that 
correspond to an item-solving strategy, and the responses from 
all examinees in a ^iven class are in accordance with a standard 
IRT model. It is further assumed that for each item, its 
parameters under the IRT model for each strategy class can be 
related to known features of the item through psychological or 
pedagogical theory. 

The next section of the paper gives a general description of 
the model. It is followed by a conceptual example that 
illustrates the key ideas. A two-stage estimation procedure is 
then presented. The first stage estimates structural parameters: 
basic parameters for test items, examinee population 
distributions, and proportions of examinees following each 
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strategy The second stage estimates posterior distributions for 
individual examinees: the probability that they belong to each 
strategy class and the conditional distribution of their ability 
for each class. A numerical example resolves examinees into 
classes of valid responders and random guessers. The final 
section discusses some iir,plications of the approach for 
educational and psychological testing. 
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The Response Model 

This section lays out the basic structure for a mixture of 
constrained item response models. Discuiision will be limited to 
dichotomous items for notational convenience, but the extensions 
to polytomous and continuous observations are straightforward. 

We begin by briefly reviewing the general form of an IRT 
model. The probability of response x^^ (1 if correct, 0 if not) 
from persoii i to item j is given by an IRT model as 



X 1 " X , 

p(x.j|<?.,/3j) - /3j)] ^j[l-f(<?^,i9j)] (1) 

where 6^ and fi^ are real (and possibly vector- valued) parameters 
associated with person i and item j respectively, and f is a 
known, twice-dif ferentiable, function whose range is the unit 
interval. Under the usual IRT assumption of local independence, 
the conditional probability of the response pattern x^^ - 

(Xj^j^ x^^) of petson i to n items is the product of n 

expressions like (1): 

o 10 
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n 

p(x.|^.,^) - n p(x |^.,)8.). 

j-i "-^ ^ J 

It may possible to express item parameters as functions of 
some smaller number of more basic parameters a - (aj^,...,a^) that 
reflect the effects of M salient characteristics of items; i.e., 

" (") • An important example of this type is the Linear 
Logistic Test Model (LLTM; Fischer, 1973, Schieblechner , 1972). 
Under the LLTM, the item response function is the one-parameter 
logistic (Rasch) model, or 

Pl^ijki.^j(a)] - exp(x.j(^..)9^))/(l+exp(^..^j)), 
and the model for item parameters is linear: 

M 

m~l ^ 

The elements of a are contributions to item difficulty associated 
with the M characteristics of item^, presumably related to the 
number or nature of processes required to solve them. The 
elements of the knowra vector indicate the extent to which item 
j exhibits each characteristic. Fischer (1973), for example, 
models the difficulty of the items in a calculus test in terms of 
the number of times an item requires the application of each of 
seven differentiation rules. Q^^ is the number of times that rule 
m must be employed in order to solve Item j. 
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Consider now a set of items that may be answered by means of 
K different strategies. It need not be the case that all are 
equally effective, nor even that all generally lead to correct 
responses. Not all strategies need be available to all 
examinees. We make the following assumptions. 



1. .^ach examinee is applying the same one of these strategies 
for all the ite.^5 in the set, (In the final section, we 
discuss prospects for relaxing this assumption to allow for 
strategy- switching) . 

2, The responses of an examinee are observed bu*: the strategy 
he or she has employed is not. 

3. The responses of examinees following Strategy k conform to 
an item response model of a known form, 

4, Substantive theory posits relationships between observable 
features of items and the probabilities of success enjoyed by 
members of each strategy class. The relationships may be 
known either fully or only partially (as when the Q matrices 
in LLTM-type rodels are known but the basic parameters ar2 
not) . 
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Let the k'th element in the K- dimensional vector 6. take the 

1 

value one if examinee i follows Strategy k, and zero if not. 
Extending the notation introduced above, we may write the 
conditional probability of response pattern as 

P(x.U..^..a) =n{n[fj^(^.j^.^ );'"'iJ)^ik 

k J 

where fi^y^^fi^^(a) gives the item parameter(s) for Item j under 
Strategy k. 

It will be natural in certain applications to partition basic 
parameters for items in accordance with strategy classes; that is, 
a « (a^, . . . ,a^) . When there are K versions of the LLTM, for 
example, differences among strategies are incorporated into the 
model by K different vectors Q , k«l,...,K, that relate Item j to 
each of the strategies: 

The item difficulty parameter for Item j under Strategy k, then, 
is a weighted sum of elements in a^, the basic parameter vector 
associated with Strategy k; the weights Qjj^ indicate the degree 
to which each of the features m, as relevent under Strategy k, are 
present in Item j. This situation will be illustrated in the 
following example. 

J.3 
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Example 1: Alternative strategies for spatial tasks 
The items of certain tests intended to measure spatial 
visualization abilities admit to solution by nonspatial analytic 
strategies (French, 1965; Kyllonen, Lohman, and Snow, 1984; 
Pelligrino, Mumaw, and Shute, 1985). Consider items in which 
subjects are shown a drawing of a three-dimensional target 
object, and asked whether a stimulus drawing could be the same 
object after rotation in the plane of the picture. In addition 
to rotation, one or more key features of the stimulus may differ 
from the those of target. A subject may solve the item either by 
rotating the target mentally the required degree and recognizing 
the match (Strategy 1), or by employing analytic reasoning to 
detect feature matches without performing rotation (Strategy 2). 

Consider further a hypothetical three-item test comprised of 
such items. Each item will be characterized by (1) rotational 
displacement, of 60, 120, or 180 degrees, and by (2) the number of 
features that must be matched. Table 1 gives the features of the 
items in the hypothetical test. 



Insert Table 1 about here 



Each subject i will be characterized by two vectors. In the 
first, « ^^il*^i2^* ^ik ^^^^^ value 1 if Subject i employs 
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Strategy k and 0 if not. In the second, 6. - (9.^,9.r^). 0., 
characterizes the proficiency of Subject i if he employs Strategy 
k. Only one of the elements of 0^ is involved in producing 
Subject i's responses, but we do not know which one. 

Suppose that for subjects employing a rotational strategy, 
probability of success is given by the one-parameter logistic 
(Rasch) item response model: 

P(x.jj5.^,^j^,<^^=l) - exp[x.j(5.^-^^^)]/[l+exp(6l.^.^j^)] . 

Here B is the proficiency of Subject i at solving tasks by 
means of the rotational strategy, and ^j^^ is the difficulty of 
Item j under the rotational strategy. 

It is usually found that the time required to solve mental 
rotation tasks is linearly related to rotational displacement. To 
an approximation, so are log-odds of success (Tapley and Bryden, 
1977). We assume that under the rotational strategy, item 
parameters take the following form: 



^jl " Qjll "11 "12 ■ 



where Q.^^^ encodes the rotational displacement of Item j--l for 



60 
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degrees, 2 for 120 degrees, and 3 for 180 degrees- -and a^^^^ is the 
incremental increase in difficulty for each increment in rotation; 
and is a constant term, for which a coefficient Qj]^2"""'" 
implied f^r all items. If ctj^j^""! and ^]^2""'^* item parameters 
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j3jj^ that are in effect under Strategy 1 are as shown in the 
second column of Table 2. 

Insert Table 2 about here 



A Rasch model will also be assumed for subjects employing 
Strategy 2, the analytic strategy, but here the item parameters 
depend on the number of features that must be matched: 

" ^j21 "21 "22 ' 

where Qj2i number of salient features, 02^^ is the 

incremental contribution to item difficulty of an additional 
feature, is a constant term, and Qj22"^ implicitly for all 
items. If a2j^'=1.5 and 022""" 2. 5, we obtain the item parameters 
that are in effect under Strategy 2. They appear in the third 
column of Table 2. 

Note that the items have been constructed so that items that 
are relatively hard under one strategy are easy under the other. 
Strategy choice cannot be inferred from obseirved response patterns 
unless patterns are more likely under some strategies and less 
likely under others. 

The response pattern Oil, for example, has a correct answer 
to an item that is easy under the Strategy 2 but hard under 
Strategy 1, and an incorrect answer to an item that is hard under 

ERIC j6 
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Strategy 2 but easy under Strategy 1. Figure 1 plots the 
likelihood function for the response vector Oil under both 
strategies; that is, p[x=(011) | ^^^,^^^-1] for k-1,2 as a function of 
0^ and $2 respectively. The maximum of the likelihood under 
Strategy 2 is about eight times as high as the maximum attained 
under Strategy 1. 



Insert Figure 1 about here 

We can make probabilistic statements about individual 
subjects if we know the proportions of people who choose each 
strategy, or tz^ = p(^j^=l), and the distributions of proficiency 
of those using each strategy class, or &y^(^y) " PC^j^l^j^-l) • 
Suppose that (i) 8^ and 0^ both follow standard normal 
distributions among the subjects that have chosen to follow them, 
and (ii) three times as many subjects follow Strategy 1 as follow 
Strategy 2--i.e., tt^ = 3/4 and t:^ = 1/4. This joint prior 
distribution is pictured in Figure 2. 

Insert Figure 2 about here 

Routine application of Bayes theorem then yields the joint 
posterior density function for <f> and k«l,...,K: 

p(5^-5,0j^-l|x,7r,a) a p [x|^j^-l , 5 ,^j^(a) ] g^(0) , (3) 
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where 

p[x|0j^«l,^)9j^(a)] = n exp{x (5-^ (a)])/{l+[5-^ (a)]) . 

j J J J 

The reciprocal of the constant of proportionality required to 
normalize (3) is the marginalization of the right side, or 

k 

The posterior distribution induced by (Oil) is shown in Figure 3. 
Marginalizing with respect to 6^ amounts to summing the area under 
the curve for Strategy k, and gives the posterior probability that 
^j^=l--that is, that the subject has employed Strategy k. The 
resulting values for this response pattern are P(^j^-1 jx-Oll)-. 28 
and P(^2^l[x«011)«.72. The prior probabilities favoring Strategy 
1 have been revised substantially in favor of strategy 2. The 
conditional posterior for given ^^^-1 has a mean and standard 
deviation of about .32 and .80. Corresponding values for the 
distribu';ion of 6^ given ^^-1 are .50 and .81. 



Insert Figure 3 about here 



Parameter Estimation 
This section discusses estimation procedures for mixtures of 
IRT models. A two-stage procedure is described. The first stage 
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integrates over 8 and ^ distributions to obtain a so-called 
marginal likelihood function for the structural parameters of the 
problem- -the basic parameters for items, the proportions of 
subjects employing each strategy, and the parameters of the B 
distributions of subjects employing each strategies. Maximum 
likelihood estimates are obtained by maximizing this likelihood 
function. If preferred, Bayes modal estimates can be obtained by 
similar numerical procedures by multiplying the likelihood by 
prior distributions for the structural parameters. The second 
stage takes the resulting point estimates of structural parameters 
as known, and calculates aspects of the posterior distribution of 
an individual examinee e. g. , p(^j^-ljx) and p(5j^j^j^-l,x) . 

Stage 1: Estimates of Structural Parameters 

Equation 2 gives the conditional probability of the response 
vector X given d and ^, or p(x|5,(^,a). Consider a population in 
which strategies are employed in proportions tt^^ and within- 
strategy proficiencies have densities Sj^C^i^kj^) among the 
examinees using them. The marginal probability of x for an 
examinee selected at random from this population is 



p(x|a,7r,,y) - S tt^^ / p(x| 5j^,<^j^«l ,a) gj^C^hj^) . (4) 

k 



For brevity, let ^ denote the extended vector of all structural 
parameters, namely (a,7r,f;). The loglikelihood for ^ induced by 
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the observation of the response vectors X - (x^,...,x^) of N 
subjects is a constant plus the sura of the logs of terms like (4) 
for each subject: 

N 

A « S log p(x lO 
i=l ^ 

- S S ^.j^ log / p[xj^j^,^j^-l,^j^(a)] gi,(\IV 
i k 

+ S S <f>., log TT . (5) 



Let S be the vector of first derivatives, and H the matrix of 
second derivatives, of A with respect to ^. Under regularity 

A 

conditions, the maximum likelihood estimates ^ solve the 
likelihood equation S=0, and a large-sample approximation of the 
matrix of estimation errors is given by the negative inverse of H 

A 

evaluated at ^. 

A standard numerical approach to solving likelihood 
equations is to use some variation of Newton's method. Newton- 
Raphson iterations, for example, improve a provisional estimate 
by adding ..he correction term -H*"'* S ^^^0- Fletcher -Powell 
iterations avoid computing and inverting H by using an 
approximation of H""*" that is built up from changes in S from one 
cycle to the next. 

These solutions have the advantage of rapid convergence if 
starting values are reasonable- -often fewer than 10 iterations 
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are necessary. S and H can be difficult tc work out, however, 
and all parameters must be usually be dealt with simultaneously 
because the off -diagonal elements in H needn't be zero. For these 
reasons, a computationally simpler but slower-converging solution 
based on Dempster, Laird, and Rubin's (1977) EM algorithm will nou 
be described as well. The approximation uses discrete 
representations for the g^^s, so the relatively simple "finite 
mixtures" case obtains (Dempster, Laird, and Rubin, 1977) 

Suppose that for each k, subject proficiency under Strategy k 
can take only the L(k) values \i » • • • »\L(k) * density g^^ is 

thus characterized by these points of support and by the weights 
associated with each, gi^(\^l\). Define the subject variable i/}^ 

= (^ill ^iKL(K)^' ^ vector of length ^ L(k) where the element 

^iki "'" proficiency of Subject i under Strategy k is 9, . 

and 0 if not. There are a total of K Is in rb., one for each 
strategy- -though again, only only of them is involved in producing 
x.--the one associated with the strategy that Subject i happens to 
employ. Summations replace integrations in the loglikelihood, 
which can now be written as 



f ^ ^ik ^ \^\2K^ 



+ S S ^ log ;r . (6) 
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If valHies of 9>and \& were\observed along with values of x, ML 
estimatiion of ^ /f rom (6) would be^impler. The basic parameter a 

( C ) 

appears ovily\4jn Jfhe first term on th^ right side of (6), so that 
maximizing v/ich respect to a need addrea^that term only. When a 
consists of di.<:tinct subvectors for each strategy, even these 
subvectors lead to distinct maximization problems of lower order. 
The subpopulation parameters appear in only che second term, 
separating them ?,n ML estimation; they too lead^to even smaller 
separate subproblems if r/ consists of distinct subvectors^^r-^^ach 
strategy. The population proportions ;r appear in only the last 
term. Unless they are fur the?" constrained, their ML estimates are 
simply observed proportions. The values of 9 may be either 
specified a priori (as in Mislevy, 1986) or estimated from the 
data (as in de Leeuw and Verhelst, 1986). In the latter case, 
their likelihood equations have contributions .from both the first 
and second terms, but the equations for the points of support 
under Strategy k involve data from only those subje-^.ts using 
Strategy k. Their aross second derivatives with points 
corresponding to other strategies are zero, although their cross 
derivatives with elements of a and that are involved with the 
same strategy need not be. 

The M-step of an EM solution requires solving a maximization 
problem of exactly this type, with one exception: the unobserved 
values of each 4>^ and yj)^ are replaced by their conditional 
expections given and provisional estimates of ^, say ^ . The 
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E-step calculates these conditional expectations as follows. 
Denote by 1^^^^ the following term in the marginal likelihood 
associated with Subject i, Strategy k, and proficiency value 9^^^ 
within Strategy k: 

The required conditional expectations are obtained as 

4i " 



and 



4 = E(^.jx..^»?0) 
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The EM formulation makes it clear how each subject 
contributes to the estimation of the parameters in all strategy 
classes, even though it is assumed that only one of them was 
relevant to the production of his responses. His data contribute 
to estimation for each strategy class is in the proportion to the 

. P.3 
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probability that that strategy was the one he employed, given his 
observed response pattern. 

In addition to its simplicity, the EM solution has the 
advantage of being able to proceed from even very poor starting 
values. The slowness with which it converges can be a serious 
drawback, however. Its rate of convergence depends on how well x 
determines examinees' 0 and ^ values. Accelerating procedures 
such as those described by Ramsay (1975) and Louis (1982) can be 
used to hasten convergence. 

Stage 2: Posteriors for Individual Examinees 

When the population parameters ^ are accurately estimated, 
the posterior density of the parameters of examinee i is 
approximately 

P^^k""^'^ik""^l''i'^^ " P[x.|\-l,^i9l^(a)] . 

where the reciprocal of the normalizing constant is obtained by 
first integrating the expression on the right over $ within each 
k, then summing over k. The posterior probability that Subject i 
used Strategy k is approximated by 
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The examinee's posterior mean and variance for a given strategy 
class, given that that was the strategy employed, are approximated 
by 

and 

If the discrete approximation has been employed, (7) and (8) 
apply. 

Example 2: A Mixture of Valid Responders and Random Guessers 
Given appropriate instructions, examinees will omit 
multiple-choice test items when they don't know the answei::^ 
rather than guess at random. The Rasch model may provide a good 
fit to such data if omits are treated as incorrect. If a small 
percentage of examinees responds at random to all items, however, 
their responses will bias the estimation of the item parameters 
that pertain to the majority of the examinees. 

We may posit a two-class model, under which an examinee 
responds either in accordance with the Rasch model or guesses 
totally at random. For examinees in the latter class, 
probabilities of correct response are constant, e.g., at the 
reciprocal of the number of response alternatives to each item. 
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Using the procedures described in the preceding sections, it is 
possible to free estimates of the item parameters that pertain to 
T:he valid responders from biases due to random guessers, even 
though it is not known with certainty who the guessers are. 

A mixture model for the (marginal) probability of response 
pattern x in this situation is 

2 

P(X.|^) - S P<X.|^^-1,0 TT^ , 

k-1 

where Strategy Class 1 is the Rasch model and Class 2 is random 
guessing. The composition of ^ is now described. It includes 
first the strategy proportions and n^. For the Rasch class, the 
basic parameters a^^ are item difficulty parameters b^ for 
j-l,...,n. Suppose the distribution g^^ of proficiencies of 
subjects following the Rasch model is discrete, with L points of 

support 9 - (9^ 9j^) and associated weights w - (w^ , . . . ,a)^) , 

The (marginal) probability of response pattern x under Strategy 1 
is 

P(x|^5j^-l,aj^,9,a)) - S o)^ H exp[x (9^.b ))/[l+exp(9jj-b )) . 

Under the random guessing strategy, the basic parameters are 
the probabilities c^ of responding correctly to each item j . All 
subjects following this strategy are assumed to have the same 
probabilities of correct response, so no distribution g^ enters 
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the picture. For such subjects, the probability of response 
pattern x is simply 

X. 1-x. 
?(x\<f>^^l,a^) = n Cj J (1-Cj) J . 



An artificial dataset was created for four items under this 
model in accordance with the following specifications. Of 1200 
simulees in all, 1000 followed the Rasch model and 200 were random 
guessers , implying and 7r2=.167. The Rasch item 

parameters were s (b^ b^) = (- .511, - .105, .182, .405) . A 

discrete density with six points of support was used to create the 
data for the Rasch class. The points and their corresponding 
proportions were as follows : 

Point Proportion 

-1.204 .08 
-.357 .17 
.095 .25 
.262 .25 
.470 .17 
.642 .08 

The rates of correct response for the random guessers on the four 
items were ^ (Cj^,...,c^) = (.30, .35, .20, .15). The 
probability of each of the sixteen possible response patterns was 
calculated within each class, multiplied by the number of simulees 
in that class, summed over classes, and rounded to the nearest 
integer. The resulting data are shown in Table 3. 
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Insert Table 3 about here 



A standard Rasch model was first fit to the data using the 
two-step marginal maximum likelihood procedures described by de 
Leeuw and Verhelst (1986). Conditional maximum likelihood (CML) 
estimates were first obtained for item parameters. Setting their 
scale by centering them around zero like the true item parameters 
for the Rasch class, the resulting values were (-.324, -.053, 
.127, .252). Note that these values are biased toward their 
center; the presence of random guessers blurs the distinctions 
among the differences in item difficulties. A three-point 
discrete distribution-- the greatest number of points leading to an 
identified model for a four- item test- -was next estimated for 
subjects. The expected counts of response patterns under this 
model are also shown in Table 3. A chi- square of 7.16 with 8 
degrees of freedom results, indicating an acceptable fit for a 
sample of the size we have employed. 

A mixture model of the generating form was then fit to the 
data, with two exceptions. First, the multiplicative form of the 
Rasch model was employed during calculations. Since maximum 
likelihood estimates are invariant under transformations, the 
estimates of the structural parameters obtained under the 
multiplicative form need merely be transformed back to the usual 
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additive form shown above. Second, a three-point discrete 
distribution was again employed for the Rasch class, with the 
lowest point fixed at zero in the multiplicative scale. This 
cojuresponds to 9^^ « -co in the additive scale, implying incorrect 
responses to all items with probability one. (As it turns out, 
the estimated weight associated with this point will be zero.) 
The total number of parameters to be estimated, then, was 13: 



2 free points in the Rasch distribution: 82 and 9^. 

2 free values for weights at the three points in the Rasch 

distribution: u>^, u>^, and w^* where S = 1. 

4 item parameters for the Rasch class: a^«(bj^, . . . ,b^) . 

4 item parameters for the guessing class: a^^^c^y . . . ,c^) . 

1 relative proportion for class representation: 7^^. 



In light of the fact that only 15 degrees of freedom are 
available from the data, in the form of 16 response patterns whose 
counts that must sum to 1200, an unaccelerated EM solution 
converged painfully slowly. Fletcher-Powell iterations were 
employed instead, and they converged rapidly. The Rasch-only 
estimates described above vjere used as starting values for the 
Rasch class item parameters and population distribution. For the 
c's, a common value midway among the true values was used. For 
TTg, starting values of .10, .15, and .20 were used in three 
different runs. All runs converged to the same solution: 
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a 



1 



- (-.501,-.091,.193,.398); 



e 



- -.534, .354); 



« (<10 



-10 



.319, .681); 



" (.287, .230, .179, .139); 



t:^ - .164. 

Although the c's are slightly underestimated, the structure of 
the data has been reconstructed quite well. The expected counts 
of response patterns are also shown in Table 3. As they should, 
they yield a nearly perfect fit: a chi-square of .008 on 3 degrees 
of freedom. The improvement in chi- square is dramatic if not 
signif icant--it would be for larger samples or longer tests--but 
the removal of the bias in the Rasch item parameter estimates is 
the point of the exercise. 

Table 4 shows conditional likelihoods of each response 
pattern given that an examinee is a guesser, a member of the 
Rasch class with 5--. 534, and a member of the Rasch class with 
^-.354. The estimated proportions of the population in these 
categories are .164, .267, and .569 respectively. Multiplying 
these population probabilities times a pattern's corresponding 
likelihood terms, then normalizing, gives the posterior 
probabilities that also appear in the table. Posterior 
probabilities are given for membership in the guessing class, and 
for ^=-.534 and ^=.354 given membership in the Rasch class. 
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Insert Table 4 about here 



Recall from the description of the EM solution that the data 
from an examinee is effectively distributed among strategy 
classes to estimate the item parameters within that class. This 
means that the responses of all examinees play a role in both 
estimating both b's and c's--but with weights in proportion to the 
posterior probabilities shown in Table 4. From responses to only 
four items , we never have overwhelming evidence that a particular 
examinee is a guesser. Only those with all incorrect responses 
can be judged more likely than not: to have guessed. Had only 
those respondents been treated as guessers- -and that would be the 
Bayesian modal estimate of strategy class- -estimated c's would all 
have been zero. But employing a proportion of data from all 
patterns, even those with all items correct, yields estimated c's 
that essentially recover the generating values. 

As a consequence of using the Rasch model for Strategy 1, 
the conditional posterior distributions given that a subject 
belongs to this class, or p(5|x,^^=l), are identical for all 
response patterns x with the same total score. The probability 
that an examinee belongs to the Rasch class vary considerably 
within patterns with the same score, however. For any given 
response pattern, the posterior probability of being in the Rasch 
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class can be inferred from Table 4 as 1 - PC^^"!!^)- patterns 
with exactly one correct response, these probabilities are, for 
Items 1-4 in turn, .869, .800, .687, and .519. 

Discussion 

Theories about the processes by which examinees attempt to 
solve test items play no role in standard applications of test 
theory, including conventional item response theory.- Only a data 
matrix of correct and incorrect responses is addressed, and items 
and examinees are parameterized strictly on the basis of 
propensities toward correct response. When all that is desired is 
a simple comparison of examinees in terms of a general propensity 
of this nature, IRT models suffice and in fact offer many 
advantages over classical true-sco" a test theory. 

Situations for which standard IRT models prove less 
satisfactory involve a desire either to better understand the 
cognitive processes that underlie item response, or to employ 
theories about such processes to provide more precise or more 
valid measurement. Extensions of item response theory in this 
direction are exemplified by the Linear Logistic Test Model 
(Schieblechner, 1972; Fischer, 1973), Embretson's (1985) 
multicomponent models, Samejima'a (1983) model jfor multiple 
strategies, and Tatsuoka's (1983) "rule space" analyses. 

The approach offered in this paper concerns situations in 
which different persons may choose different strategies from a 
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If items were constructed to run from easy to hard under the 
rotational strategy and hard to easy under the analytic, subjects 
using the third "mixed" strategy would find them easy, then 
harder, then easier again. 

There are limitations to how far these ideas can be pressed 
in applications with binary data. Our second example showed that 
the misspecified Rasch model fit a four-item test acceptably well 
with a sample of 1200 subjects; in one way or another, more 
information would be needed to attain a sharper distinction 
between strategy classes and, correspondingly, more power to 
differentiate among competing models for the data. One source of 
information is more binary items. Fifty items rather than four, 
including some that are very hard under the Rasch strategy, would 
do. A different source of information available in other settings 
would be to draw from richer observational possibilities. 
Examples would include response latencies as well as correctness, 
eye-fixation patterns, and choices of incorrect alternatives that 
are differentially likely under different strategies. 

Differentiating the likelihood of response patterns under 
different strategies is the key to successful applications of the 
approach. Its use would be recommended when identifying strategy 
classes is of primary importance to the selection or placement 
decision that must be made And overall proficiency is of 
secondary importance. The items in the test must then be 
constructed to maximize strategy differences, e.g., using items 
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that are hard under one strategy but easy under another. Most 
tests in current use with standard test theory are not constructed 
with this purpose in mind; indeed, they are constructed so as to 
minimize differentiation among strategies, since it lowers the 
reliability of overall-propensity scores. When strategy class 
decisions are of interest, a conventional tests is not likely to 
provide useful information. (Although a battery of conventional 
tests might; differences in score profiles are analogous to 
differential likelihoods of item response patterns, but at a 
higher level of aggregation.) 

In addition to the applications used in the preceding 
examples, a number of other current topics in educational and 
psychological research are amenable to expression in terms of 
mixtures of IRT models. We conclude by mentioning three. 

Hierarchical development. Wilson's (1984, 1985) "saltus" 
model (Latin for "leap") extends the Rasch model to developmental 
patterns in which capabilities increase in discrete stages, by 
including stage parameters as well as abilities for persons, and 
stage parameters as well as difficulties for items. Examples 
would include Piaget's (1960) innate developmental stages and 
Gagne's (1962) learned acquisition of rules. Suppose that K 
stages are ordered in terms of increasing and cumulative 
competence. In our notation, <j> would indicate the stage 
membership of a subject. In the highest stage, item responses 
follow a Rasch model with parameters b.. Rasch models fit lower 
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stages as well, but the item parameters are offset by amounts that 
depend on which stage tho item can first be solved. Our basic 
parameters a would correspond to the item parameters for the 
highest stage and the offset parameters for particular item types 
at particular lower stages. Figure 4 gives a simple illustration 
in which items associated with higher stages have an additional 
increment of difficulty for subjects at lower stages. In 
applications such as Siegler's (1981) balance beam tasks, subjects 
at selected lower stages tend to answer certain types of higher- 
stage items correctly for the wrong reasons. In these cases, the 
offset works to give easier item difficulty parameters to those 
items in those stages. 



Insert Figure 4 about here 



Mental models for problem solving. In the introduction to 
their experimental study on mental models for electricity, 
Centner and Centner (1983) state 

Analogical comparisons with simple or familiar systems often 
occur in people's descriptions of complex systems, sometimes 
as explicit analogical models, and sometimes as implicit 
analogies, in which the person seems to borrow structure from 
the base domain without knowing it. Phrases like "current 
being routed along a conductor" and "stopping the flow" of 
electricity are examples (p. 99). 

Mental models are important as a peaagogical device and as a 
guide to problem- solving. Inferring which models a person is 
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using, based on a knowledge of how conceivable analogues help or 
hinder the solution of certain types of problems, provides a 
guide to subsequent training. In Centner and Centner's 
experiment, the problems concerned simple electrical circuits with 
series and parallel combinations of resistors and batteries. 
Popular analogies for electricity are flowing waters (S'trategy 1) 
and "teeming crowds" of people entering a stadium through a few 
narrow turnstiles (Strategy 2). The water flow analogy 
facilitates battery problems, but does not help with resistor 
problems; indeed, it suggests an incorrect solution for the 
current in circuits with parallel resistors. The teeming crowd 
analogy facilitates problems on the combination of resistors, but 
is not informative about combinations of batteries. If a Rasch 
model holds for items within strategies. Centner and Centner's 
hypotheses correspond to constraints on the order of item 
difficulties with the two strategics. If each item type were 
replicatated enough times, it would be possible to make inferences 
about which model a particular examinee was using, in order to 
plan subsequent instruction. 

Changes in intelligence over agct. An important topic in the 
field of human development is whether, and how, intelligence 
changes as people age (Birren, Cunningham, and Yamamoto, 1983). 
Macrae (n.d.) identifies a v/eakness of most studies that employ 
psychometric tests to measure aging effects: total scores fail to 
reflect important differences in the strategies different subjects 
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bring to bear on the items they are presented. Total score 
differences among age and educational -background groups on Raven's 
matrices test were not significant in the study she reports. But 
analyses of subjects' introspective reports on how they solved 
items revealed that those with academically oriented background 
were much more likely to have used the preferred "algorithmic" 
strategy over a "holistic" strategy than those with vocationally 
oriented backgrounds. Since the use of algorithmic strategies was 
found to increase probabilities of success differentially on 
distinct item types, this study would be amenable to IRT mixture 
modeling. Inferences could then be drawn about problem-solving 
approaches without resorting to more expensive and possibly 
unreliable introspective evidence. 
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Table 1 
Item Features 

Item rotational displacement salient features 

1 60 degrees 3 

2 120 degrees 2 

3 180 degrees 1 



Table 2 
Item Difficulty Parameters 

Item Strategy 1 Strategy 2 

1 -1.0 2.0 

2 0.0 0.5 

3 1.0 -1.0 
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Table 3 

Observed and Fitted Response Pattern Counts for Example 2 



X 


observed 
frequencies 


expected frequencies 
(Rasch model only) 


expected frequencies 
(2-class model) 


A AAA 
UUUU 




143 


.00 


143 


.08 


0001 


94 


98 


.66 


93 


.95 


0010 


83 


87 


.12 


83 


11 


0011 


101 


90 


.55 


101 


09 


0100 


73 


72 


.75 


72 


78 


0101 


78 


76 


.62 


77 


75 


0110 


65 


66 


.77 


65 


26 


0111 


106 


93 


.20 


105 


98 


1000 


64 


55 


.46 


63 


91 


1001 


54 ■ 


57 


65 


54 


16 


1010 


47 


50 


91 


46 


75 


1011 


71 


71 


06 


70 


94 


1100 


39 


42 


51 


39 


30 


1101 


54 


59 


34 


54 


07 


1110 


45 


52 


40 


44. 


80 


1111 


83 


83 


00 


83. 


07 
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Table 4 

Response Pattern Likelihoods and Posterior Probabilities 



X L(x|^2) L(x|e2,^^) L(x|e2,^^) P(<1>2\^) P(e2|x,^^) P(e3|x,^^) 



0000 


.388 


.150 


.027 


.534 


.719 


.281 


0001 


.063 


.131 


.058 


.131 


.513 


.487 


0010 


.085 


.107 


.047 


.200 


.513 


.487 


0011 


.014 


.093 


.100 


.027 


.303 


.697 


0100 


.116 


.080 


.036 


.313 


.513 


.487 


0101 


.019 


.070 


.076 


.047 


.303 


.697 


Olio 


.025 


.057 


.062 


.0''6 


.303 


.6^7 


0111 


.004 


.050 


.131 


.008 


.151 


.849 


1000 


.156 


.053 


.024 


.481 


.513 


.487 


1001 


.025 


.047 


.050 


.092 


.303 


.697 


1010 


.034 


.038 


.041 


.143 


.303 


.697 


1011 


.005 


.033 


.087 


.015 


.151 


.849 


1100 


.047 


.029 


.031 


.234 


.303 


.697 


1101 


.008 


.025 


.065 


.027 


.151 


.849 


1110 


.010 


.020 


.053 


.045 


.151 


.849 


1111 


.002 


.018 


.113 


.004 


.068 


.932 



Note: <{>^ denotes membership in the 
<f>2 denotes membership in the 
9^ denotes membership in the 

with ^"-.534; 
9^ denotes membership in the 

with ^-.354. 



class of Rasch responders; 
class of random guessers; 
class of Rasch responders, 

class of Rasch responders, 
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likelihood function 



0.45 




-4. -3 -2 -1 0 1 2 3 4. 

theta 
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Figure 1 
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prior distribution 

0.40 -| — 

0.35 - 




Figure 2 
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posterior distribution 

0.40 -I , 




-4-3-2-1 0 1 2 3 4 

the to 

o p(theta-i,psi1!01 1) + p(theta2»psi2|01 1 ) 

Figure 3 
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1 2 3 4 5 6 

Item difficulties- 
highest stage 



1 2 3 4 5 6 



Item diff icultier- 
middle stage 



1 2 3 4 5 6 



Item difficulties- 
lowest stage 



Figure 4 

Saltus example: 3 stages, common offset 
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2100 Second Street, S.W. 
Washington, DC 20593 
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American College Testing 
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Chapel Hill, NC 27514 
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Washington, DC 20370 
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Dr. Norman CI iff 
Department of Psychology 
Univ. of So. Cal ifornia 
Uni versity Park 
Los Angeles, CA 90007 

Di rector, 

Manpower Support and 

.^ead i ness Program 
Center for Naval Analysis 
2000 North Beauregard Street 
Alexandria, VA 22311 

Dr. Stanley Col Iyer 
Office of Naval Technology 
Code 222 

800 N. Quincy Street 
Arl ington, VA 22217-5000 

Dr. Hans Crombag 
University of Leyden 
Education Research Center 
Boerhaave I aan 2 
2334 EN Leyden 
The NETHERLANDS 

Mr. Timothy Oavey 
Uni versi ty of 1 1 I inoi s 
Educati ona I Psycho I oyy 
Urbana, IL 61801 

Dr. C. M. Dayton 
Department of Measurement 

Statistics & Evaluation 
Col I ege of Education 
University of Maryland 
Col lege Park, MD 20742 

Dr. Ralph J. DeAyala 
Measurement, Stat i st i cs, 

and Evaluation 
Benjami n Bu i I d i ng 
University of Maryland 
Col leg<5 Park, MD 2C742 

Dr. Dattprasad Divgi 
Center for Naval Analysis 
4401 Ford Avenue 
P.O. Box 16268 
Alexandria, VA 22302-0268 



Dr. Hei-Ki Dong 

Bell Communications Research 

b Corporate P I ace 
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Piscataway, NJ 08854 

Dr. Fritz Drasgow 
University of III inois 
Department of Psychology 
603 E. Daniel St. 
Champaign, IL 61820 

Defense Technical 

Information Center 
Cameron Station, BIdg 5 
Alexandria, VA 22314 
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Dr. Stephen Dunbar 
L indquist Center 

for Measurement 
University of Iowa 
Iowa City, I A 52242 

Dr. James A. Karles 

Air Force Human Resources Lab 

Brooks AF8, TX 78235 

Dr. Kent Eaton 
Army Research Institute 
6001 Eisenhower Avenue 
Alexandria, VA 22333 

Dr. John M. Eddins 
University of 1 1 I inois 
252 Engineering Research 

Laboratory 
103 South Mathews Street 
Urbana, IL 61801 

Dr. Susan Embretson 
University of Kansas 
Psychology Department 
426 F raser 
Lawrence, KS 66045 

Dr. George Englehard, Jr. 
Division of Educational Studies 
Emory University 
201 F ishburne BIdg. 
Atlanta, 6A 30322 
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Dr. Benjamin A. Fairbank 
Performance Metrics, Inc. 
5825 Cal laghan 
Suite 225 

San Antonio, TX 78228 

Dr. Pat Kederi co 
Code 511 
NPRDC 

San Diego, CA 92152-6800 

Dr. Leonar d Fe I dt 
L i ndqu i st Center 

for Measurement 
University of Iowa 
Iowa City, I A 52242 

Dr. Richard L. Ferguson 
American College Testing 

Program 
P.O. Box 168 
Iowa City, I A 52240 

Dr. Gerhard Fischer 
Liebiggasse 5/3 
A 1010 Vienna 
AUSTRIA 

Dr. Myron FischI 
Army Research Institute 
5001 Eisenhower Avenue 
Alexandria, VA 22333 

Prof. Donald Fitzgerald 
University of New England 
Department of Psychology 
Armidale, New South Males 2351 
AUSTRALIA 

Mr. Paul Foley 

Navy Personnel R&D Center 

San Diego, CA 92152-6800 

Dr. Alfred R. Fregly 
AFOSR/NL 

Bol I ing AFB, DC 20332 

Dr. Robert D. Gibbons 

Illinois State Psychiatric Inst. 

Rm 529M 

1601 M. Taylor Street 
Chicago, IL 60612 
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Dr. Janice Gifford 
University of Massachusetts 
School of Education 
Amherst, MA 01003 

Dr. Robert Glaser 
Learning Research 

& Development Center 
University of Pittsburgh 
3939 O'Hara Street 
Pittsburgh, PA 15260 

Dr. Bert Green 
Johns Hopkins University 
Department of Psychology 
Charles & 34th Street 
Baltimore, MD 21218 

Dipl. Pad. Michael M. Habon 
Universitat Dusseldorf 
Erziehungswissenschaf tl iches 
Universitatsstr. 1 
D-4000 Dusseldorf 1 
WEST GERMANY 

Dr. Ronald K. Hambleton 

Prof, of Education & Psychology 

University of Massachusetts 

at Amherst 
Hills House 
Amherst, MA 01003 

Dr. Delwyn Harnisch 
Un i vers i ty of 1 1 I i no i s 
51 Gerty Drive 
Champaign, IL 61820 

Ms. Rebecca Hetter 

Navy Personnel R&D Center 

Code 62 

San Diego, CA 92152-6800 

Dr. Paul M. Holland 
Educational Testing Service 
Rosedale Road 
Princeton, NJ 08541 

Prof. Lutz F. Hornke 
Institut fur Psychologie 
RMTH Aachen 
Jaegerstrasse 17/19 
D-5100 Aachen 
WEST GERMANY 
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Dr. Paul Horst 
677 G Street, #184 
Chula Vista, CA 90010 

Mr. Dick Hoshaw 
OP- 135 

Ar I i ngton Annex 
Room 2834 

Washington, DC 203bO 

Dr. Lloyd Humphreys 
University of 1 1 I inois 
Department of Psychology 
603 East Daniel Street 
Champaign, IL 61820 

Dr. Steven Hunka 
Department of Education 
University of Alberta 
Edmonton, Alberta 
CANADA 

Dr. Huynh Huynh 
Col I ege of Education 
Univ. of South Carolina 
Columbia, SC 29208 

Dr. Robert Jannarone 
Department of Psychology 
University of South Carolina 
Columbia, SC 29208 

Dr. Dennis E. Jennings 
Department of Statistics 
University of 1 1 I inois 
1409 West Green Street 
Urbana, IL 61801 

Dr. Douglas H. Jones 
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Army Research Institute 
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Prof. John A. Keats 
Department of Psychology 
University of Newcastle 
N.S.W. 2308 
AUSTRALIA 

Dr. G. Gage Kingsbury 

Portland Public Schools 

Research and Evaluation Department 

501 North Dixon Street 

P. 0. Box 3107 

Portland, OR 97209-3107 

Dr. Wi I I i am Koch 
University of Texas-Austin 
Measurement and Evaluation 

Center 
Austin, TX 78703 

Dr. James Kraatz 
Computer-based Education 

Research Laboratory 
Un i vers i ty of Illinois 
Urbana, IL 61801 

Dr. Leonard Kroeker 

Navy Personnel R&D Center 

San Diego, CA 92152-6800 

Dr. Dary I I Lang 

Navy Personnel R&D Center 

San Diego, CA 92152-6800 

Dr. Jerry Lehnus 

Defense Manpower Data Center 

Suite 400 

1600 Wi Ison Blvd 

Rosslyn, VA 22209 

Dr. Thomas Leonard 
University of Wisconsin 
Department of Statistics 
1210 West Dayton Street 
Madison, Wl 53705 

Dr. Michael Levine 
Educational Psychol ogy 
210 Education BIdg. 
University of II I inois 
Champaign, IL 61801 
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Dr. Charles Lewis 
Educational Testing Service 
Princeton, NJ 08541 

Dr. Robert Linn 
Co I I ege of Educat i on 
)n i vers i ty of 1 1 11 no i s 
drbana, IL 61801 

Dr. Robert Lockman 
Center for Naval Analysis 
4401 Ford Avenue 
P.O. Box 16268 
Alexandria, VA 22302-0268 

Dr. Frederic M. Lord 
Educational Testing Service 
Princeton, NJ 08541 

Dr. Mi I ton Ma i er 
Center for Naval Analysis 
4401 Ford Avenue 
P.O. Box 16268 
Alexandria, VA 22302-0268 

Dr. Mi I I jam L. Maloy 
Chief of Naval Education 

and Training 
Naval Air Station 
Pensacola, FL 32508 

Dr. Gary Marco 
Stop 31-t 

Educational Testing Service 
Princeton, NJ 08451 

Dr. CI essen Mart i n 
Army Research Institute 
5001 Eisenhower Blvd. 
Alexandria, VA 22333 

Dr. James McBride 
Psychological Corporation 
c/o Harcourt, Brace, 

Javanov i ch Inc. 
1250 West 6th Street 
San Diego, CA S2101 

Dr. Clarence McCormick 

HQ, MEPCOM 

MEPCT-P 

2500 Green Bay Road 
North Chicago, IL 60064 
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Dr. George B. Macready 
Department of Measurement 
Statistics & Evaluation 
College of Education 
University of Maryland 
Col lege Park, MD 20742 

Or. Robert McKinley 
Educational Testing Service 
20-P 

Princeton, NJ 08541 

Dr. James McMichael 
Techni cal Di rector 
Navy Pe.'^sonnel R&U Center 
San Oieyo, CA 92152 

Dr. Barbara Means 
Human Resources 

Rese.T* ^^h Organization 
llOO So. w Washington 
Alexandria, VA 22314 

Dr. Robert Mi s I evy 
Educational le-oting Service 
Princeton. NJ 08541 

Or. Wi I I iam Montagu'^ 

NPROC Code 13 

San Diego. CA 92152-6800 

Ms. Kathleen Moreno 

Navy Personnel R&O Center 

Code 62 

San Diego, CA 92152-6800 

Headquarters, Marine Corps 
Code MPl-20 
Washington, DC 20380 

Dr. W. Alan Nicewander 
University of Oklahoma 
Department of Psychology 
Oklahoma City, OK 73069 

Deputy Technical Director 

NPROC Code OlA 

San Diego, CA :i52-6800 

Director Training Laboratory, 

NPROC (Code 05) 
San Diego, CA 92152-6800 
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Director, Manpower and Personnel 

Laboratory, 

NPRDC (Code 06) 
San Diego, CA 92152-6800 

Director, Human Factors 

& Organizational Systems Lab, 

NPRDC (Code 07) 
San Diego, CA 92152-6800 

Fleet Support Office, 

NPRDC (Code 301) 
San Diego, CA 92152-6800 

Library, NPRDC 
Code P201L 

San Diego, CA 92152-6800 

Commanding Officer, 

Naval Research Laboratory 
Code 2627 

Washington, DC 20390 

Dr. Harold F. O'Nei I, Jr. 
School of Education - WPH 801 
Department of Educational 

Psychology & Technology 
University of Southern California 
Los Angeles, CA 90089-0031 

Dr. James Olson 
WICA1, Inc. 

1875 South State Street 
Orem, UT 84057 

Office of Navai Research, 

Code 1142CS 
800 N. Quincy Street 
Arl ington, VA 22217-5000 
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Office of Naval Research, 

Code 125 
800 N. Quincy Street 
Arlington, VA 22217-5000 

Assistant for MPf Research, 
Development and Studies 
OP 0187 

Washington, DC 20370 
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Dr. Judith Orasanu 
Army Research Institute 
5001 Eisenhower Avenue 
Alexandria, VA 22333 

Dr. Jesse Orlansky 
Institute for Defense Analyses 
1801 N. Beauregard St. 
Alexandria, VA 22311 

Dr. Randolph Park 
Army Research Institute 
5001 Eisenhower Blvd. 
Alexandria, VA 22333 

Wayne M. Pat i ence 
American Council on Education 
GED Testing Service, Suite 20 
One Dupont Ci rcl e, NW 
Washington, DC 20036 

Dr. James Paulson 
Department of Psychology 
Portland State University 
P.O. Box 751 
Portland, OR 97207 

Administrative Sciences Department, 

Naval Postgraduate School 
Monterey, CA 93940 

Department of Operations Research, 

Naval Postgraduate School 
Monterey, CA 93940 

Dr. Mark D. Reckase 
ACT 

P. 0. Box 168 

Iowa City, lA 52243 « 

Dr. Malcolm Ree 
AFHRL/MP 

Brooks AFB, TX 78235 

Dr. Barry Riegelhaupt 
HumRRO 

1100 South Washington Street 
Alexandria, VA 22314 

Dr. Carl Ross 
CNEl-PDCD 
Bui I ding 90 

Great Lakes NIC, IL 60088 
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Dr. J. Ryan 

Department of Education 
University of South Carolina 
Columbia, SC 29208 

Dr. Fumiko Samejima 
Department of Psychology 
University of Tennessee 
3108 AustinPeay BIdg. 
Knoxvi Me, IN 37916-0900 

Mr. Drew Sands 

NPRDC Code 62 

San Diego, CA 921S2-6800 

Lowel I Schoer 

Psychological & Quantitative 

Foundat i ons 
Co I I ege of Educat i on 
University of Iowa 
Iowa City, I A 52242 

Dr. Mary Schratz 

Navy Personnel R&D Center 

San Diego, CA 92152-6800 

Dr. Dan Segal I 

Navy Personnel R&D Center 

San Diego, CA 92152 

Dr. M. Steve Se I Iman 

OASD(MRA&L) 

2B269 The Pentagon 

Washington, DC 20301 

Dr. Kazuo Shigemasu 
7-9-24 Kugenuma-Ka i gan 
Fujusawa 251 
JAPAN 

Dr. Mi I I i am Sims 
Center for Naval Analysis 
4401 Ford Avenue 
P.O. Box 16268 
Alexandria, VA 22302-0268 

Dr. H» Mai lace Sinaiko 
Manpower Research 

and Advisory Services 
Smi thson i an Inst i tut ion 
801 North Pitt Street 
Alexandria, VA 22314 
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Dr. Richard E. Snow 
Department of Psychology 
Stanford Un i vers i ty 
Stanford, CA 94306 

Dr. Richard Sorensen 
Navy Personnel R&D Center 
San Diego, CA 92152-6800 

Dr. Paul Speckman 
University of Missouri 
Department of Statistics 
Columbia, MO 65201 

Dr. Judy Spray 
ACT 

P.O. 3ox 168 

Iowa City, I A 52243 

Dr. Martha Stocking 
Educational Testing Service 
Princeton, NJ 08541 

Dr. Peter Stoloff 
Center for Naval Analysis 
200 North Beauregard Street 
Alexandria, VA 22311 

Dr. Wi I I iam Stout 

Un i vers i ty of 1 1 I i no i s 

Department of Statistics 

101 mini Hal I 

725 South Mrighi St. 

Champaign, IL 61820 

Maj. Bill Strickland 
AF/MPXOA 
4E168 Pentagon 
Mashington, DC 20330 

Dr. Harihsran Swaminathan 
Laboratory of Psychometric and 

Evaluation Research 
School of Education 
University of Massachusetts 
Amherst, MA 01003 

Mr. Brad Sympson 

Navy Personnel R&D Center 

San Diego, CA 92152-6800 
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Dr. John Tangney 
AFOSR/NL 

Boll ing AKB, DC 20332 

Dr. Kikumi Fatsuoka 
CERL 

252 Engineering Research 

Laboratory 
Urbana, IL 61801 

Dr. Maurice Fatsuoka 
220 Education BIdg 
1310 S. Sixth St. 
Champaign, IL 61820 

Dr. David Fhissen 
Department of Psychology 
University of Kansas 
Lawrence, KS 66044 

Mr. Gary Thomasson 
University of 1 1 I inois 
Educat i ona I Psycho I ogy 
Champaign, IL 61820 

Dr, Robert Tsutakawa 
University of Missouri 
Department of Statistics 
222 Math. Sciences BIdg. 
Columbia, MO 6b211 

Dr. Ledyard Fucker 
University of 1 1 I inois 
Department of Psychology 
603 E. Daniel Street 
Champaign, IL 61820 

Dr. Ve^rn W. Urry 

Personnel R&D Center 

Office of Personnel Management 

1900 E. Street, NW 

Washington, DC 20415 

Dr. David Vale 
Assessment Systems Corp. 
2233 University Avenue 
Suite 310 
St. Paul, MN 55114 

Dr. Frank V i c i no 

Navy Personnel R&D Center 

San Diego, CA 92152-6800 



Dr. Howard Nainer 
Division of Psychological Studies 
Educational Testing Service 
Princeton, NJ 08541 

Dr. Ming-Mei Wang 
L i ndqu i st Center 

for Measurement 
University of Iowa 
Iowa City, lA 52242 

Dr. Thomas A. Warm 
Coast Guard Institute 
P. 0. Substation 18 
Oklahoma City, OK 73169 

Dr. Bri an Waters 
Program Manager 
Manpower Analysis Program 
HumRRO 

1100 S. Washington St. 
Alexandria, VA 22314 

Dr. Dav id J. Weiss 
N660 El I iott Hal I 
University of Minnesota 
75 E. River Road 
Minneapol is, MN 55455 

Dr. Ronald A. Weitzman 
NPS, Code 54Wz 
Monterey, CA 92152-6800 

Major John We I sh 
AFHRL/MOAN 

Brooks AFB, TX 78223 

Dr. Douglas Wetzel 
Code 12 

Navy Personnel FJ&D Center 
San Diego, CA 92152-6800 

Dr. Rand R. Wi Icox 
University of Southern 

Cal ifornia 
Department of Psychology 
Los Angeles, CA 90007 
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German Military Representative 
ATTN: Wolfgang Wildegrube 

Streitkraefteamt 

D-5300 Bonn 2 
4000 Brandywine Street, NW 
Washington, DC 20016 

Dr. Bruce Willi ams 
Department of Educational 

Psycho I ogy 
Un i vers i ty of Illinois 
Urbana, IL 61801 

Dr. Hi Ida Wing 
NRC GF-176 

2101 Constitution Ave 
Washington, DC 20418 

Dr. Martin F. Wiskoff 
Navy Personnel R&D Center 
San Diego, CA 92152-6800 

Mr. John H. Wolfe 
Navy Personnel R&D Center 
San Diego, CA 92152-6800 

Dr. George Wong 
Bi ostat i st i cs Laboratory 
Memori a I SI oan-Ketter i ng 

Cancer Center 
1275 York Avenue 
New York, NY 10021 

Dr. Wal lace Wulf eck. III 
Navy Personnel R&D Center 
San Diego, CA 92152-6800 

Dr. Kentaro Yamamoto 
Computer-based Education 

Research Laboratory 
Un i vers i ty of 1 1 I i no i s 
Urbana, IL 61801 

Dr. Wendy Yen 
CTB/McGraw Hill 
Del Monte Research Park 
Monterey, CA 93940 

Dr. Joseph L. Young 
Memory & Cogn i t i ve 

Processes 
National Science Foundation 
Washington, DC 20550 58 
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